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1.0  EXECUTIVE  SUMMARY 

1.1  BACKGROUND 

The  detection  of  Buried  Unexploded  Ordnance  (UXO)  is  one  of  the  most  serious  environmental 
problems  facing  the  Department  of  Defense.  In  addition,  false  alarms  due  to  clutter  are  also  a 
serious  problem.  With  traditional  survey  methods,  the  Army  Corps  of  Engineers  finds  that  85- 
95%  of  all  detected  targets  are  not  UXO.  Since  the  cost  of  identifying  and  disposing  of  UXO  in 
the  United  States  using  current  technologies  is  estimated  to  range  up  to  $500  billion,  increases  in 
performance  efficiency  due  to  reduced  false  alarm  rates  can  result  in  substantial  cost  savings. 
The  investments  of  SERDP,  ESTCP  and  the  U.S.  Army  Environmental  Center  UXO  Advanced 
Technology  Demonstration  Programs  in  automated  UXO  detection  and  discrimination 
technologies  have  resulted  in  site  characterization  technologies  that  have  demonstrated  detection 
probabilities  of  greater  than  95  percent.  However,  there  are  always  buried  targets  that  are  too 
small,  or  are  buried  too  deeply,  to  be  detected  with  current  systems.  The  Jefferson  Proving 
Ground  (JPG)  Phase  I-III  demonstrations  indicate  improved  system  performance,  but  not 
necessarily  an  improvement  in  the  size-depth  detection  envelope. These  detection  results 
reflect  the  performance  of  experienced  operators  selecting  anomalies  from  displays  of  survey 
data.  Studies  of  visual  signal  detection  consistently  show  that  human  operators  do  not  perform 
as  well  as  the  optimal  linear  processor.  Human  observer  efficiency  for  detection  of  aperiodic 
signals  in  white  noise  is  about  50%  and  is  further  reduced  with  the  addition  of  structured  noise. ^ 
Reported  detection  efficiencies  in  structured  backgrounds  are  10%  for  detection  of  disk  signals 
in  simulated  lumpy  backgrounds  consisting  of  Gaussian-shaped  blobs,^  4%  to  17%  for  detection 

o 

of  simulated  lesions  in  backgrounds  representative  of  coronary  angiograms,  and  36%  for 
detection  of  a  vehicle  in  natural  settings.^  However,  optimal  linear  filtering  has  not  been  used 
previously  for  UXO  detection.  Such  a  processor  is  based  on  thresholding  the  output  of  a 
prewhitening  matched  filter'®  that  compensates  for  any  background  correlation  structure 
(prewhitens)  and  then  correlates  the  data  with  the  expected  signal.  It  is  optimal  in  that  it 
maximizes  the  output  signal  to  noise  ratio,  and  is  variously  referred  to  in  the  pattern  recognition 
and  medical  imaging  literature  as  the  Eisher  linear  discriminant,  the  Hotelling  observer  or  the 
Eisher-Hotelling  model.  If  the  noise  statistics  are  Gaussian,  it  corresponds  to  the  maximum 
likelihood  detector  and  maximizes  probability  of  detection." 

In  this  project,  we  have  developed  a  Matched  Eilter  AutoProcessor  (MEAP)  that  implements  an 
optimal  linear  filter  in  a  threshold-based  UXO  detection  processor.  It  also  implements  screening 
procedures  based  on  target  attributes  (e.g.,  size,  depth,  orientation)  determined  from  their 
magnetic  signatures,  in  order  to  discriminate  between  UXO  and  the  false  alarms  due  to  metallic 
clutter.  The  individual  screening  criteria  compare  the  various  target  attribute  values  with 
threshold  levels  set  by  the  user.  The  results  of  the  individual  tests  are  combined  using  Boolean 
logic,  allowing  the  user  to  specify  which  attributes  to  include  in  the  logical  sum.  It  is  expected 
that  the  MEAP  will  provide  improved  detection  performance  for  weak/deep  targets  in 
magnetically  active  or  noisy  areas.  The  detection  and  classification  modules  can  be  added  to 
existing  detection  and  analysis  systems.  In  addition  to  improving  detection  of  targets  in 
magnetically  noisy  areas,  the  matched  filter  also  reduces  ambiguity  in  the  location  of  a  target, 
because  the  filter  output  peaks  directly  over  the  target  location  (unlike  raw  magnetometer  data, 
for  which  the  center  of  the  dipole  pattern  is  not  typically  at  the  target  location). 
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The  Matched  Filter  AutoProcessor  was  developed  and  tested  using  the  Interactive  Data 
Language  (IDL).  This  report  discusses  details  of  the  algorithm  and  presents  performance 
measures.  Upon  successful  completion  and  acceptance  of  the  development  effort,  the  codes 
were  rewritten  in  C++  language  to  create  a  dynamic  link  library  that  interfaces  with  the  OASIS 
Montaj  processing  environment.  OASIS  Montaj  is  developed  and  marketed  commercially  by 
Geosoft,  Inc.  Appendix  B  presents  a  discussion  of  OASIS  Montaj  integration  effort.  The 
Matched  Filter  software  (matchfilter.gx  and  matchlilter.dll)  and  documentation  (Matched  Filter 
ReadMe.doc)  are  available  for  download  via  the  ESTCP  website. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  objective  of  the  demonstration  was  to  show  whether  the  MFAP  improves  UXO  detection 
and  false  alarm  rejection  relative  to  procedures  currently  used  in  the  field,  particularly  for 
weak/deep  targets  in  magnetically  active  or  noisy  areas.  The  demonstration  of  this  automatic 
processor  was  performed  on  magnetometer  data  collected  during  the  more  recent  2000  UXO 
Detection/Discrimination  Advanced  Technology  Demonstration  at  the  Jefferson  Proving  Ground 
in  Madison,  Indiana  (henceforth  called  JPG  Technology  Demonstration,  or  JPG  TD).  This 
demonstration  took  place  on  three  1 -hectare  sites  that  contained  both  inert  UXO  and  typical 
ordnance  scrap  specifically  seeded  into  a  magnetically  active  geological  environment.  The  JPG 
TD  demonstrators  (who  all  used  active  EM  systems)  were  asked  to  detect,  locate  and  identify 
UXO  in  the  three  areas,  producing  a  target  list  with  recommendations  for  each  target  whether  the 
item  should  be  dug  or  could  confidently  be  left  in  the  ground.'^  The  demonstration  of  this 
matched  filter  autoprocessor  used  magnetometer  array  data  collected  over  the  same  sites  and 
followed  the  same  requirements,  to  the  extent  that  the  magnetometer  (rather  than 
electromagnetic)  data  and  this  technology  allow.  The  ground  truth  of  the  JPG  Technology 
Demonstration  was  used  to  determine  the  performance  of  the  matched  filter  processor. 
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1.3  DEMONSTRATION  RESULTS 


The  curves  below  are  the  Receiver  Operating  Characteristics  (ROCs)  for  the  primary 
demonstrators  at  JPG  TD,  who  used  electromagnetic  induction  (EMI)  systems.  The  three 
colored  points  on  each  plot  indicate  the  relative  performance  of  analysis  of  magnetometer  data 
using  the  MFAP  and  U-Hunter,  as  well  as  standard  Mag  and  Flag  methodology.  U-Hunter  and 
Mag  and  Flag  points  generally  lie  far  to  the  right  of  the  curves,  indicating  significantly  poorer 
performance.  The  MFAP  points,  on  the  other  hand,  reveal  performance  comparable  to  the  best 
demonstrators  in  Area  2,  and  intermediate  between  the  various  demonstrators  in  Areas  1  and  3. 
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2.0  TECHNOLOGY  DESCRIPTION 


2. 1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

Buried  Unexploded  Ordnance  (UXO)  is  one  of  the  most  serious  and  prevalent  environmental 
problems  currently  facing  Department  of  Defense  facility  managers.  SERDP,  ESTCP  and  the 
U.S.  Army  Environmental  Center  UXO  Advanced  Technology  Demonstration  Programs  have 
been  addressing  the  need  for  automated  UXO  detection  and  discrimination  technologies  for 
several  years.  These  investments  have  resulted  in  technologies  that  have  demonstrated  UXO 
detection  probabilities  of  greater  than  95  percent.  The  detection  results  reflect  the  performance 
of  experienced  operators  selecting  anomalies  from  displays  of  survey  data.  Studies  of  visual 
signal  detection,  notably  in  the  medical  imaging  community,  consistently  show  that  human 
operators  do  not  perform  as  well  as  the  optimal  linear  processor.  ’  ’  ’  The  optimum  linear 
detection  processor  maximizes  the  output  signal  to  noise  ratio,  and  if  the  noise  statistics  are 
Gaussian,  it  corresponds  to  the  maximum  likelihood  detector  and  maximizes  probability  of 
detection.'^ 

In  addition  to  detection  of  UXO,  false  alarms  due  to  clutter  (signals  incorrectly  diagnosed  as 
having  been  caused  by  UXO)  remain  a  serious  problem.  The  Army  Corps  of  Engineers  finds 
that  85-95%  of  all  targets  detected  with  traditional  survey  methods  are  not  UXO.  Since  the  cost 
of  identifying  and  disposing  of  UXO  in  the  United  States  using  current  technologies  is  estimated 

to  range  up  to  $500  billion,  increases  in  performance  efficiency  due  to  reduced  false  alarm  rates 
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can  result  in  substantial  cost  savings.  ’ 

The  Matched  Eilter  AutoProcessor  (MEAP)  we  have  developed  is  an  optimum  linear  detection 
processor.  It  is  expected  that  this  processor  will  provide  improved  detection  performance  for 
weak  or  deep  targets  in  magnetically  active  or  noisy  areas.  The  MEAP  also  applies  Boolean 
logic  to  target  attribute  screening,  in  order  to  rationalize  the  UXO/clutter  discrimination  process 
and  reduce  false  alarms.  The  signatures  of  clutter  items  that  cause  false  alarms  can  be  subtly 
different  from  the  signatures  of  UXO  items,  and  experienced  operators  do  some  screening  of  the 
field  anomalies  to  eliminate  obvious  false  alarms.  Anomaly  characterization  procedures  in  the 
Data  Analysis  System  developed  by  AETC  for  use  with  MTADS  estimate,  for  each  detected 
object,  various  target  attributes  from  the  measured  magnetic  anomaly.  Systematic  and  objective 
target  screening  criteria  for  the  MEAP  are  based  on  these  attributes. 

The  detection  and  classification  modules  can  be  incorporated  into  the  existing  Multisensor 
Towed  Array  Detection  System  {MTADS)  Data  Analysis  System  developed  by  NRL,  or  other 
systems,  and  serve  in  the  future  as  a  testbed  for  exploring  site-specific  optimization  of  threshold 
settings  and  decision  rules  for  UXO  detection  and  false  alarm  rejection.  The  processor’s  target 
selection  routine  should  provide  more  automatic  and  thus  efficient  target  selection  and  thus 
lower  expenses  for  large  survey  areas.  In  addition,  future  versions  of  the  MEAP  can  be  modified 
to  analyze  active  electromagnetic  system  data  as  well  as  passive  magnetometer  data. 
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2,2  PROCESS  DESCRIPTION 


In  general,  a  matehed  filter  operates  by  convolving  the  data  with  an  appropriate  signal  model, 
accompanied  by  a  search  over  the  unknown  parameters  in  the  signal  model  to  maximize  the  filter 
output.  Detections  are  called  when  the  filter  output  exceeds  some  threshold.  This  threshold  can 
be  varied  to  achieve  the  desired  performance.  The  MFAP  operates  on  magnetic  survey  data  and 
uses  a  filter  based  on  a  magnetic  dipole  at  a  given  depth.  It  maximizes  the  filter  output  at  each 
assumed  target  horizontal  location  with  a  search  over  size  and  orientation  of  the  magnetic 
moment.  The  matched  filter  output  is  enhanced  at  locations  where  the  data  fits  the  model  well. 
In  other  words,  UXO  whose  depths  are  close  to  the  chosen  fdter  depth  will  show  up  in  the 
filtered  data  as  peaks  standing  out  above  the  background.  The  filter  output  near  objects  that  are 
not  dipole-like  (e.g.,  geologic  features  that  may  have  strong  signatures  in  the  raw  data)  or  are  not 
compact  (e.g.,  magnetic  soil)  will  be  suppressed. 

Since  the  depths  of  the  expected  targets  are  unknown,  the  filtering  procedure  is  repeated  over 
several  filter  depths.  We  have  observed  that  deep  targets  ring  out  strongly  over  a  relatively  wide 
range  of  filter  depths,  so  the  filter  depths  can  be  spaced  at  half-meter  or  even  one-meter  intervals. 
For  each  filtered  depth,  the  image  created  from  the  matched  filter  output  is  contoured  and  peaks 
are  identified  as  provisional  (dipole-like)  targets.  Multiple  targets  at  the  same  location  selected 
from  several  depths  are  reduced  to  a  single  provisional  target.  The  horizontal  location  of  each 
provisional  target  is  then  input  to  a  dipole-fitting  routine  whose  outputs  include  fitted  dipole 
depth,  size,  orientation,  and  coherence  (a  goodness  of  fit  parameter). 

Since  some  objects  that  are  not  UXO  (e.g.,  metallic  clutter  or  magnetic  rocks)  can  also  give 
compact  dipole  signals  and  be  identified  as  provisional  targets,  we  have  implemented  screening 
procedures  based  on  the  target  attributes  determined  from  their  magnetic  signatures,  in  order  to 
reduce  false  alarms.  The  individual  screening  criteria  are  based  on  comparison  of  the  target 
attribute  values  with  threshold  levels  set  by  the  user.  The  results  of  the  individual  tests  are 
combined  using  Boolean  logic,  allowing  the  user  to  specify  which  attributes  to  include  in  the 
logical  sum. 

The  four  processing  steps  are  summarized  below.  More  detail  on  the  processing  can  be  found  in 
the  project  final  report. 

Step  1:  Filter  the  magnetic  data 

•  the  magnetometer  data  is  mapped  and  interpolated  to  a  regular  x-y  grid 

•  the  operator  selects  a  set  of  target  depths  for  the  matched  filter,  based  upon  the 
expected  types  of  ordnance  and  their  maximum  depths 

•  for  each  depth,  the  matched  filter  runs  automatically  on  the  data,  also  removing 
any  local  linear  magnetic  background  in  the  data  as  it  filters 

The  choice  of  the  filter  depth  is  not  critical.  The  filter  creates  a  filter  box  around  each  data  point 
in  the  image  to  perform  the  dipole  fit  and  background  subtraction  on  the  data.  The  size  of  the 
filter  box  increases  with  increasing  filter  depth,  in  order  to  capture  the  increasing  width  of  a 
dipole’s  signature  at  the  sensor.  As  currently  implemented,  the  algorithm  cannot  filter  a  data 
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point  if  there  are  any  missing  data  in  the  filter  box.  Thus,  the  edges  of  a  survey  site  (half  a  filter 
box  width)  eannot  be  filtered,  as  well  as  any  interior  regions  that  are  missing  raw  data  (e.g., 
beeause  of  poor  navigation  or  obstacles  in  the  survey  field).  For  a  filter  depth  of  0.5  m,  a 
(typical)  strip  of  missing  magnetometer  data  0.5  x  5  m  (2.5  m^)  becomes  an  unfilterable  region 
5.4  X  9.9  m  (52.9  m  ).  For  larger  filter  depths,  this  problem  becomes  more  severe.  However,  the 
increased  area  lost  at  deeper  depths  overstates  the  number  of  potential  targets  lost.  Because  the 
majority  of  deep  targets  also  show  up  in  images  filtered  at  shallower  depths,  a  deep  target  that 
lies  in  a  data  gap  at  a  large  filter  depth  may  he  outside  that  gap  at  a  shallower  depth. 

Step  2:  Identify  peaks  in  the  matched  filter  output  as  provisional  targets 

•  for  each  target  depth  chosen,  the  filter  output  is  displayed  as  an  image 

•  the  operator  selects  a  detection  threshold  and  an  automatic  peak-selector  detects 
threshold  exceedances  in  the  output  field 

The  peaks  in  the  filter  output  image  are  found  with  a  contouring  threshold  procedure.  The 
operator  can  use  the  default  threshold  value,  derived  from  the  cumulative  distribution  function 
(CDF)  of  the  filter  output,  or  select  a  different  threshold  if  the  default  results  show  missed  targets 
that  are  obvious  strong  dipoles,  or  if  there  are  too  many  weak  and  possibly  spurious  targets 
selected.  Our  tests  on  vehicular  MTADS  magnetometer  data  from  several  sites  showed  that  a 
threshold  value  equal  to  the  filter  output  for  a  CDF  of  0.75  performed  satisfactorily.  Filtered 
data  is  contoured  with  levels  lying  between  the  threshold  and  the  maximum  filtered  data  level. 
Maximum  points  within  concentric  contours  are  identified  as  peaks  or  provisional  targets.  After 
provisional  targets  are  selected  from  each  filter  depth,  duplicate  targets  (chosen  at  a  location  for 
more  than  one  filter  depth)  are  removed  in  order  to  get  the  final  list  of  provisional  targets. 

Step  2:  Fit  provisional  targets  to  magnetic  dipole  model 

•  the  x-y  location  of  each  provisional  target  is  input  to  a  dipole  fitting  routine  that 
estimates  the  best  dipole  fit  from  the  data 

•  the  targets  are  tabulated  with  the  dipole-fitting  output:  fitted  dipole  depth, 
moment,  radius,  orientation  and  data/model  coherence  (a  goodness-of-fit 
parameter  that  ranges  from  0  to  1.0.) 

Step  4:  Screen  and  sort  the  provisional  targets  on  the  basis  of  expected  target  attributes, 
combining  the  individual  screening  results  with  Boolean  logic 

•  targets  are  eliminated  which  do  not  fit  a  dipole  model  well 

•  targets  are  eliminated  whose  fitted  sizes/depths  are  too  large/small  to  be  UXO 

•  any  additional  target  attributes  expected  for  the  particular  site  or  type  of  ordnance 
is  used  for  screening 

•  the  remaining  targets  are  ordered  by  closeness  of  fitted  dipole  moment  angle  to 
the  Earth’s  field 

•  the  operator  selects  a  dipole  moment  angle  threshold  to  separate  UXO  from 
clutter 
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The  data/model  coherence  is  an  important  screening  parameter.  The  dipole  contribution 
dominates  the  magnetic  signature  of  a  UXO  item  at  any  sensible  range.  Some  clutter  items  (e.g., 
buried  sheet  metal,  sections  of  wire),  on  the  other  hand,  are  expected  to  exhibit  significant 
departures  from  the  dipole  model  and  thus  have  a  lower  data/model  coherence  value.  The 
threshold  on  coherence  is  selected  by  determining  the  coherence  values  that  detect  all  obvious 
strong  dipoles  in  the  data  without  adding  large  numbers  of  clearly  spurious  targets.  In  general, 
we  find  a  coherence  threshold  of  0.8  to  be  adequate.  The  MFAP  removes  objects  from  the 
provisional  target  list  that  (a)  are  too  weak  to  determine  their  fit  to  a  dipole  model,  and  (b)  fit  a 
dipole  model  with  coherence  less  than  the  selected  threshold. 

Our  dipole-fitting  procedures  reliably  estimate  target  depth  to  within  less  than  10%.'"^  This  can 
be  used  to  eliminate  objects  that  are  deeper  than  the  deepest  UXO  expected  for  the  survey  area. 
Much  of  the  clutter  is  due  to  small  metal  fragments  at  or  near  the  surface,  and  can  often  be 
rejected  by  screening  on  the  basis  of  target  size.  Target  size  estimates  are  not  as  accurate  as 
depth  estimates,  since  for  typical  UXO  the  strength  of  the  magnetic  signal  (and  thus  derived  size) 
varies  somewhat  with  the  orientation  of  the  target.  Our  experience  with  UXO  of  various  sizes, 
buried  at  various  depths  and  orientations,  is  that  the  size  estimates  have  a  standard  deviation  of 
about  30%  around  the  actual  ordnance  caliber,  which  will  usually  suffice  for  distinguishing 
between  shell  fragments  and  intact  UXO.^^  It  will  also  suffice  to  screen  out  objects  that  are  too 
large  or  small  to  be  UXO  of  the  types  expected  for  the  survey  area. 

Dipole  orientation  can  also  be  a  useful  screening  parameter.  At  least  some  clutter  items  have  a 
different  distribution  of  dipole  orientations  than  that  of  UXO Ordnance  items  tend  to  have 
low  remnant  magnetization  due  to  their  construction  (cast  as  opposed  to  rolled  or  drawn)  and/or 
history  (the  shock  effects  of  firing  and  impact  with  the  ground),  so  their  dipole  moments  are  due 
solely  or  mostly  to  the  Earth’s  field.  For  all  possible  orientations  of  a  demagnetized  ordnance 
item  there  is  a  maximum  possible  angle  0max  between  the  induced  dipole  moment  and  the  Earth’s 
magnetic  field;  this  angle  is  determined  by  the  aspect  ratio  of  the  ordnance.  Clutter  items,  on  the 
other  hand,  can  have  significant  remnant  magnetic  moments,  resulting  in  a  broader  distribution 
of  dipole  orientations;  their  dipole  moments  can  be  found  at  all  possible  angles  to  the  Earth’s 
field.  This  difference  in  dipole  moment  orientation  is  used  in  the  MFAP  to  reject  as  clutter  all 
objects  with  moments  oriented  at  angles  to  the  Earth’s  field  larger  than  Bmax-  (Note  that  both 
measurement  error  and  the  types  of  UXO  expected  in  the  survey  area  need  to  be  taken  into 
account  when  0max  is  determined.)  Among  the  objects  with  angles  less  than  0max  one  expects  to 
find  all  the  ordnance,  as  well  as  some  of  the  clutter. 

Additional  target  attributes  or  correlations  between  attributes  can  also  be  used  as  screening 
criteria  as  they  become  recognized  with  more  data  analysis.  For  the  demonstration  on  JPG  TD 
data,  as  described  in  section  4.1,  an  additional  screening  criterion  was  added  to  two  of  the  survey 
areas  after  the  ground  truth  from  the  one  of  the  areas  was  received  and  reviewed:  the  added 
criterion  was  the  correlation  between  the  data/model  coherence  and  the  fitted  depth. 

After  screening  of  provisional  targets  is  completed,  the  remaining  targets  can  be  ordered  by 
closeness  of  the  fitted  dipole  angle  to  the  Earth’s  field. 
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3.0  DEMONSTRATION  DESIGN 


3.1  PERFORMANCE  OBJECTIVES 

The  demonstration  of  the  MFAP  was  not  performed  in  the  field;  it  was  instead  performed  in 
eontractor  offices  using  magnetometer  array  data  previously  collected  at  Jefferson  Proving 
Ground  in  Madison,  Indiana,  during  the  2000  Advanced  UXO  Detection/Discrimination 
Technology  Demonstration  (JPG  TD)/^ 

The  performance  objectives  of  the  matched  filter  autoprocessor  are  to  improve  detection  and 
discrimination  of  UXO,  particularly  for  weak  or  deep  targets  in  magnetically  active  or  noisy 
areas.  The  performance  metrics  are  probability  of  target  detection  and  probability  of  false 
alarms. 


Table  1.  Performance  Criteria, 


Performance  Criteria 

Description 

Primary  or 
Secondary 

Hazardous  Materials 

Using  magnetometer  data,  detect  UXO  in  the 
size  range  from  20  mm  to  155  mm  projectiles. 
Determine  the  probability  of  detection. 

primary 

Reliability 

Determine  the  percentage  of  false  alarms. 

primary 

Ease  of  Use 

Determine  how  much  training  is  necessary  to 
obtain  optimal  results. 

secondary 

3.2  SITE  CHARACTERISTICS 

This  demonstration  was  performed  in  the  Arlington,  Virginia,  offices  of  AETC,  using  data  that 
was  previously  collected  on  a  government  range  (JPG)  for  other  purposes.  There  are  thus  no 
regulatory,  health  or  safety  issues  affecting  this  demonstration.  However,  the  details  of  the 
demonstration  site,  operations,  and  analysis  requirements  are  relevant  to  this  demonstration, 
since  the  results  of  the  processor  are  compared  to  the  results  of  the  JPG  demonstration. 

The  JPG  demonstration  prepared  three  1 -hectare  sites  containing  inert  UXO,  ordnance  scrap,  and 
magnetic  soils/rocks.  Two  of  the  three  sites  took  advantage  of  naturally-occurring  magnetic  soil 
deposits.  Three  demonstrators  were  invited  to  participate  using  active  electromagnetic  sensors; 
the  NRL  man-portable  EMMS,  Geophex  with  a  variant  of  the  GEM  sensor  and  NAEVA  with  the 
Geonics  EM-63  sensor.  Each  demonstrator  was  asked  to  conduct  digitally-mapped  geo- 
referenced  surveys  and  to  conduct  target  analyses  on  site  as  though  concurrent  remediation 
would  take  place.  Ten  types  of  inert  ordnance  were  included,  varying  in  size  from  20-mm 
aircraft-fired  projectiles  to  155-mm  howitzer  projectiles.  Demonstrators  were  told  that  emplaced 
inert  ordnance  was  degaussed.  The  non-UXO  items  were  only  described  as  realistic  ordnance 
scrap. 
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The  demonstrators  were  also  asked  to  produee  a  list  of  detected  targets  prioritized  as  UXO  or 
not-UXO  with  an  associated  degree  of  confidence,  and  a  recommendation  for  which  targets 
should  be  dug  and  which  could  confidently  be  left  in  the  ground.  Two  independent  analyses  and 
lists  were  to  be  submitted,  based  upon  whether  or  not  20-mm  projectiles  were  assumed  to  be 
present.  The  onsite  EM  surveys  and  analysis  products  were  to  be  evaluated  on  the  basis  of 
cost/production  rates,  UXO  detection  rates,  target  location  accuracy  and  UXO  identification 
ability. 

The  JPG  demonstrators  deployed  and  were  evaluated  on  use  of  active  electromagnetic  sensors. 
In  addition,  an  independent  survey  was  made  of  the  three  sites  using  the  NRL  vehicular  MTADS 
magnetometer  array.  These  data  were  preprocessed  to  create  geo-referenced  mapped  data  files 
that  were  provided  to  each  of  the  other  demonstrators  following  their  reporting  of  EM  results,  for 
optional  joint  magnetometer-EM  data  analyses.  This  MTADS  magnetometer  data  set  was 
analyzed  by  the  MEAP.  The  target  list  produced  by  this  analysis  was  prioritized  into  UXO  and 
not-UXO  with  confidence  categories,  in  a  manner  similar  to  the  three  JPG  TD  demonstrators. 
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4.0 


PERFORMANCE  ASSESSMENT 


4. 1  DEMONSTRATION  DATA  ANALYSIS 

The  primary  performance  criteria  for  the  Matched  Filter  AutoProcessor  (MFAP)  are  probability 
of  detection  and  probability  of  false  alarm.  A  secondary  performance  measure  is  the  ease  of 
using  the  MFAP  in  terms  of  operator  training  necessary  to  obtain  satisfactory  results.  The  data 
used  to  estimate  these  measures  is  vehicular  MTADS  magnetometer  data  from  the  JPG  TD.  The 
seeded  UXO  ranged  in  size  from  20-mm  aircraft-fired  projectiles  to  155-mm  howitzer 
projectiles.  The  non-UXO  items  buried  in  the  survey  areas  were  realistic  ordnance  scrap,  but 
were  not  described  further.  We  were  informed  that  all  ordnance  was  demagnetized  before  burial. 
Target  lists  for  the  three  JPG  TD  survey  areas  were  produced.  The  detected  targets  were  sorted 
into  categories  of  UXO  and  not-UXO,  with  confidence  levels  associated  with  each  target. 

The  vehicular  sensor  array  was  0.25  m  above  the  ground.  In  the  standard  pre-processing 

the  data  was  gridded  and  mapped  to  0.25  m  spacing  in  both  x  and  y.  All  depths  for  buried 
objects  (dipole  sources,  UXO,  clutter,  etc.)  are  referenced  to  the  surface  of  the  ground,  and  are 
positive  downward. 

Step  1:  Filter  the  magnetic  JPG  TD  data 

Figure  1  shows  raw  and  filtered  magnetometer  images  from  JPG  TD  Area  2.  The  raw  data  (on 
the  left)  shows  strong,  extended  geologic  features.  The  second  image  shows  the  filter  output  for 
a  filter  depth  of  0.10  meters.  One  can  see  that  the  filter  does  an  excellent  job  of  removing  the 
strong,  extended  geologic  features  in  the  raw  magnetometer  data  and  producing  isolated  peaks  in 
the  fdter  output. 
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The  choice  of  the  filter  depth  is  not  critical.  We  have  found  that  deep  targets  ring  out  strongly  in 
the  MFAP  output  over  a  relatively  wide  range  of  filter  depths,  as  shown  below  in  Figure  2  for 
JPG  TD  Area  3.  The  images  show  the  data  filtered  at  0.1  meter  (left)  and  2.0  meters  (right); 
deep  targets  are  clearly  visible  at  both  the  deep  and  shallow  filter  depths. 

Thus,  for  the  JPG  TD  data,  the  filtering  procedure  was  performed  with  only  5  filter  depths.  A 
shallow  filter  depth  (0.10  m)  was  used  to  identify  the  overwhelming  majority  of  both  the  shallow 
and  deep  targets.  Three  deeper  filter  depths  (1.0,  2.0,  and  3.0  m)  were  used  to  identify  additional 
deep  targets  that  were  not  clearly  visible  in  the  0.10  m  filtered  output.  The  last  (0.50  m)  filter 
depth  was  used  to  identify  any  targets  that  might  lie  between  0.10  and  1.0  m  but  were  too  small 
to  appear  unambiguously  as  targets  in  the  0.1  m  images.  (Note  that  when  we  performed  spot 
checks  by  filtering  the  data  at  other  depths,  no  additional  targets  were  found.) 


Filtered  Mag  Data,  Z  =  0.1  m,  JPGTD  Area  3 


Filtered  Mag  Data,  Z  =  2.0  m,  JPG  TD  Area  3 
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Figure  2.  Target  Locations  (Crosses)  from  JPG  TD  Area  3  Overlaid  on  Filter  Output  Images  at 
Shallow  and  Deep  Filter  Depths,  0.1  m  (left)  and  2.0  m  (right). 


Figures  1  and  2  show  that  the  filtered  images  contain  regions  without  any  matched  filter  output. 
The  MFAP  creates  a  filter  box  around  each  data  point  in  the  image  to  perform  the  dipole  fit  and 
background  subtraction  on  the  data.  The  size  of  the  filter  box  increases  with  increasing  filter 
depth,  in  order  to  capture  the  increasing  width  of  a  dipole’s  signature  at  the  sensor.  As  currently 
implemented,  the  algorithm  cannot  filter  a  data  point  if  there  are  any  missing  data  in  the  filter 
box.  Thus,  the  edges  of  a  survey  site  (half  a  filter  box  width)  cannot  be  filtered,  as  well  as  any 
interior  regions  that  are  missing  raw  data  (e.g.,  because  of  poor  navigation  or  obstacles  in  the 
survey  field).  Figure  3  shows  data  from  JPG  TD  Area  1  filtered  at  0.1  m  and  3.0  m  depths.  The 
area  lost  at  the  edges  is  10%  for  the  0.1  m  filter  depth  and  25%  for  the  3.0  m  filter  depth.  The 
area  lost  to  interior  data  gaps  is  3%  and  8%,  respectively,  for  the  two  filter  depths.  However,  the 
increased  area  lost  at  deeper  depths  overstates  the  number  of  potential  targets  lost.  As  discussed 
above,  the  majority  of  deep  targets  also  show  up  in  images  fdtered  at  shallower  depths,  so  a  deep 
target  that  lies  in  a  data  gap  at  a  large  filter  depth  may  lie  outside  that  gap  at  a  shallower  depth. 
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Figure  3.  Data  from  JPG  2000,  Area  1,  Showing  the  Increase  in  Size  of  the  Data  Gaps  with  Filter 
Depth,  from  0.1  m  Filter  Depth  (left)  to  3.0  m  Filter  Depth  (right). 


Step  2:  Identify  peaks  in  the  matched  filter  output  as  provisional  targets 

When  the  default  eontour  threshold  for  picking  targets  was  determined  from  the  global 
cumulative  distribution  function  (CDF)  of  the  filter  output  over  each  survey  area,  we  found  that  a 
huge  number  of  spurious  peaks  were  picked  over  the  entire  region,  especially  around  the  stronger 
targets.  The  shape  of  the  CDF  over  an  entire  JPG  TD  area  was  markedly  different  from  that  over 
a  small  sub-patch.  The  solution  was  to  break  each  JPG  area  into  smaller  areas  (30  m  on  a  side) 
and  perform  independent  contour  thresholding  based  on  the  local  CDF  (using  the  usual  75% 
threshold).  From  this  procedure,  a  reasonable  number  of  provisional  targets  were  identified  by 
the  autoprocessor. 

Table  2  shows  the  number  of  provisional  targets  found  in  JPG  TD  Area  2  after  filtering  at  the 
five  depths  and  removing  duplicate  targets.  The  results  for  Areas  1  and  3  were  similar. 

Table  2,  Provisional  Targets  from  JPG  TD  Area  2. 


Filter  depth  (meters) 

Number  of  Targets  Found 

Percent  of  Total  Targets 

0.1 

170 

77 

0.5 

15 

7 

1.0 

20 

9 

2.0 

11 

5 

3.0 

4 

2 

Total 

220 

too 
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Step  3:  Fit  provisional  targets  to  magnetic  dipole  model 

A  number  of  peaks  identified  by  the  matched  filter  autoprocessor  were  too  weak  to  fit  to  a  dipole 
model.  These  were  eliminated  from  the  target  list  without  further  analysis. 

Step  4:  Screen  and  sort  the  provisional  targets  on  the  basis  of  expected  target  attributes 

The  original  target  screening  attributes  used  were  data/model  coherence,  fitted  target  size,  fitted 
target  depth.  After  determining  a  cutoff  for  each  target  attribute  and  screening  the  provisional 
targets,  the  remaining  targets  were  ordered  by  closeness  of  the  fitted  dipole  orientation  to  the 
Earth’s  field.  Dig  lists  were  produced  from  these  criteria. 

However,  after  the  ground  truth  from  JPG  TD  Area  3  was  received,  the  results  were  analyzed  to 
see  if  there  should  be  any  modifications  in  or  additional  screening  criteria  for  the  final  target  lists 
for  Areas  1  and  2.  There  were  two  changes,  discussed  further  below,  made  to  the  screening 
criteria:  (1)  the  buffer  zone  in  dipole  orientation  to  the  Earth’s  field  was  removed  and  (2)  targets 
with  joint  deep  fitted  depths  and  relatively  low  model/data  coherence  were  eliminated  from  the 
target  list. 

The  data/model  coherence  cutoff  was  0.8,  as  determined  from  our  experience  with  data  from 
other  surveys.  This  removed  a  large  number  of  provisional  targets  (30%-40%).  Several  targets 
with  coherence  less  than  0.8  were  nevertheless  retained  in  the  target  list  because  there  were 
obvious  navigation  errors  in  the  data  that  produced  the  low  coherence.  (Note  that  some  of  the 
targets  in  Area  3  rejected  for  low  coherence  for  non-navigation  reasons  did  turn  out  to  be 
ordnance,  as  discussed  in  the  next  section.) 

Since,  as  previously  discussed,  size  estimates  derived  from  the  dipole  fitting  routine  have  a 
standard  deviation  of  about  30%  around  the  actual  ordnance  caliber,  we  used  a  45%  (a  1.5  a) 
rejection  window.  We  rejected  all  provisional  targets  whose  fitted  size  was  smaller  than  55%  of 
the  smallest  expected  EIXO  or  larger  than  145%  of  the  largest  expected  EIXO.  Eor  the  JPG  TD, 
these  were  20  mm  projectiles  and  155  mm  howitzers,  respectively.  The  results  of  this  screening 
rejected  no  targets  because  they  were  too  small,  and  rejected  only  1  (out  of  674  provisional 
targets  in  all  three  areas)  because  it  was  too  large. 

Objects  that  were  deeper  than  the  deepest  expected  EIXO  were  also  screened.  Only  1  target  was 
rejected  as  too  deep  for  the  JPG  data;  the  fitted  depth  for  that  object  was  greater  than  our 
threshold  of  4  meters. 

Eor  the  orientation-angle  screening  criterion,  we  calculated  the  maximum  expected  angle  (0max) 
between  the  induced  dipole  moment  of  a  non-magnetized  object  and  the  Earth’s  field.  This  angle 
0max  increases  with  increasing  aspect  ratio.  The  ordnance  in  JPG  TD  with  the  largest  ratio  was 
the  2.75  in  rocket  (aspect  ratio  6.5:1);  the  corresponding  maximum  allowed  angle  is  57°. 

Eigure  4  shows,  for  Areas  1,  2  and  3  (left  to  right),  shaded  in  red,  the  possible  azimuth- 
inclination  combinations  possible  for  a  cylinder  with  a  6.5:1  aspect  ratio  with  no  remnant 
magnetization.  The  region  has  a  sharp  boundary,  corresponding  to  the  maximum  allowed  angle 
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0max  of  57°.  Superimposed  on  this  plot  are  the  dipole-fitting  routine’s  azimuth  and  inelination 
fits  for  all  the  provisional  targets  in  the  three  areas  that  had  coherences  above  the  cutoff  of  0.80 
(blue  triangles).  Those  items  within  the  red  shaded  region  may  be  ordnance  and  therefore  should 
be  dug  up;  ordnance  should  not  be  found  outside  the  red  shaded  region. 
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Figure  4,  Possible  Azimuth-Inclinations  for  a  6.5:1  Aspect  Ratio  Cylinder  with  No  Remnant 
Magnetization,  for  Areas  1,  2,  3  (left  to  right),  (The  blue  triangles  are  the  provisional  targets  with 

coherenees  above  0.80.) 


There  are  uncertainties  in  the  dipole  fitting  routine’s  estimates  of  the  induced  dipole  moments  of 
the  provisional  targets  and  ordnance  items  may  have  some  remnant  magnetization.  Thus,  some 
ordnance  items  may  lie  outside  the  red  region  in  Figure  4.  To  account  for  these  inaccuracies,  the 
initial  decision  was  to  add  a  buffer  zone  of  23°  and  set  80°  as  the  cutoff  cone  angle  between 
possibly  ordnance  and  definitely  clutter.  The  value  of  80°  is  in  agreement  with  the  largest  angles 
we  have  observed  for  ordnance  at  other  sites.  However,  in  reviewing  the  ground  truth  results  for 
Area  3,  we  observed  that  all  of  the  ordnance  items  fell  within  O^ax,  and  none  fell  in  the  buffer 
zone  between  57°  and  80°.  We  therefore  decided  that  we  were  too  conservative  in  our  angle 
cutoff  and  we  removed  the  buffer  zone  in  the  criteria  for  Areas  1  and  2.  The  results  discussed  in 
this  report  and  the  final  dig  lists  for  Areas  1  and  2  (which  can  be  found  in  the  final  report  )  are 
without  the  buffer  zone  in  the  angle  criterion. 

A  final  screening  criterion  was  developed  from  reviewing  the  ground  truth  for  JPG  TD  Area  3. 
Figure  5  shows,  for  all  targets  in  Area  3  that  passed  all  the  screening  criteria,  their  fitted  depth 
and  the  data/model  coherence.  The  targets  that  are  truly  ordnance  are  plotted  as  blue-filled 
diamonds.  It  is  clear  that  there  are  no  UXO  with  joint  low  coherences  and  deep  fitted  depths. 

This  is  reasonable  when  it  is  considered  that  larger  projectiles  or  bombs  will  likely  bury 
themselves  deeper  in  the  ground  than  smaller  ordnance,  and  for  these  larger  objects  the  signal 
and  data/model  coherence  is  expected  to  be  stronger  than  for  clutter.  Thus,  objects  with 
coherences  below  0.93  and  depths  below  0.6  m  were  moved  into  the  dig  list  category  of  high 
confidence  clutter,  regardless  of  their  other  target  attribute  values. 
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Coherence 


Figure  5.  Fitted  Depth  and  Data/Model  Coherence  for  Targets  Selected  for  JPG  TD  Area  3,  after 
Screening,  Plotted  as  Open  Red  Diamonds.  [Targets  that  are  truly  ordnanee  (determined  from  ground 
truth)  are  blue-filled  diamonds.  The  dotted  line  divides  the  target  area  from  an  exeluded  area.] 

The  targets  that  passed  all  the  screening  criteria  above  were  then  sorted  by  the  closeness  of  the 
fitted  dipole  moment  to  the  Earth’s  field,  and  all  targets  were  given  confidence  levels  on  their 
categorization  as  ordnance  or  clutter.  The  objects  in  the  final  dig  list  above  the  “do  not  dig 
below”  line  are  those  that  fit  a  dipole  model  well,  that  were  not  bigger,  smaller  or  deeper  than  the 
expected  UXO  types,  were  not  too  deep  for  the  coherence  value,  and  whose  dipole  orientation 
could  be  induced  by  the  Earth’s  field.  However,  if  an  object  fits  all  these  criteria,  there  is  no 
objective  measure  to  discriminate  among  them,  and  the  final  dig  list  therefore  has  only  two 
categories:  (1)  high  confidence  clutter  and  (2)  ordnance-or-clutter. 

Table  3  shows  the  final  screening  results  for  JPG  TD.  Eor  Area  3,  these  results  are  based  on  the 
original  screening  parameters.  Eor  Areas  1  and  2,  the  modified  screening  based  on  information 
from  Area  3  ground  truth  were  used. 

Table  3,  Final  Target  Screening  Results  for  JPG  TD, 


Screening  Criteria 

Area  1 

Area  2 

Area  3 

total  #  of  targets  before  screening 

213 

220 

241 

rejected  (too  weak  to  fit  as  a  dipole) 

24 

36 

55 

rejected  (too  small) 

0 

0 

0 

rejected  (too  large) 

0 

1 

0 

rejected  (too  deep) 

0 

1 

0 

rejected  (data/model  coherence  <  0.8) 

60 

63 

47 

replaced  in  list  (coh  <0.8  but  nav  errors) 

6 

0 

0 

Total  #  of  targets  after  screening 

129 

119 

139 

ordnance  or  clutter  (9  <  0n,ax) 

50 

53 

91 

clutter  with  high  confidence  (0  >  9max) 

79 

66 

48 
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Note  that,  after  all  non-UXO  were  eliminated  because  of  the  criteria  on  target  attributes  of  size, 
depth,  and  data/model  coherence,  the  remaining  targets  were  ordered  by  closeness  of  fitted 
dipole  moment  direction  to  that  of  the  Earth’s  field.  Then  the  operator  can  classify  the  targets 
with  moment  angles  outside  the  maximum  expected  angle  as  high  confidence  clutter  and  the 
targets  with  moment  angles  within  the  maximum  expected  angle  can  be  classified  as  ordnance  or 
clutter.  However,  the  MFAP  analysis  of  magnetometer  (rather  than  electromagnetic)  array  data 
cannot  discriminate  further  among  these  categories. 

Figure  6  shows  the  final  filter  output  images  for  Areas  1,  2,  3  for  a  filter  depth  of  0.1  m.  The 
targets  that  were  recommended  to  dig  up  are  marked  with  crosses.  For  the  image  from  Area  3, 
the  seeded  ordnance  are  marked  with  triangles  (19  items)  and  the  seeded  clutter  are  marked  with 
diamonds  (34  items).  It  can  be  seen  that  there  are  some  seeded  clutter  items  that  were  selected  to 
dig  (10  items),  and  others  that  were  not  (24  items). 


Filtered  Mag  Data,  Z  =  0.1  m,  JPG  TO  Area  3 


Figure  6.  Filter  Output  Images  for  JPG  TD  Area  1  (left),  2  (right)  and  3  (bottom)  for  a  Filter 
Depth  of  0.1  m.  (Targets  to  dig  up  are  marked  with  crosses.  For  Area  3,  the  seeded  ordnance  are 
marked  with  triangles;  seeded  clutter  are  marked  with  diamonds.) 
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4,2  PERFORMANCE 


The  matched  filter  autoprocessor  (MFAP)  appeared  to  do  a  good  job  of  isolating  potential  targets 
in  the  magnetically  noisy  and  active  areas  encountered  in  JPG  TD.  This  was  in  agreement  with 
the  results  of  our  previous  tests  using  the  fdter  on  several  other  sites  (Laguna,  Buckley,  Blossom 
Point,  Twentynine  Palms).  The  fdter  output  peaked  up  over  potential  targets  as  expected, 
reducing  ambiguity  in  the  exact  location  of  targets.  The  data  used  for  the  matched  filter 
autoprocessor  demonstration  was  magnetometer  array  data  from  the  vehicular  MTADS  surveys  at 
JPG  TD.  The  magnetometers  were  spaced  0.25  m  apart,  so  surface  coverage  was  not  100%.  We 
expected  that  small  objects,  such  as  20-mm  projectiles,  could  pass  between  the  magnetometers 
without  being  detected,  while  large  objects  buried  more  deeply  would  more  likely  be  detected. 

The  ground  truth  for  JPG  TD  Area  3  lists  a  total  of  20  UXO  and  35  OE.  The  UXO  were: 


20  mm  projectiles  4 

57  mm,  60  mm  5 

2.75  in  1 

76  mm  1 

81  mm  4 

105  mm  1 

152  mm  1 

155  mm.  1 


Table  4  shows  the  detection  dig  list  results  for  Area  3.  Of  the  20  seeded  UXO,  the  MFAP  dig  list 
contained  16  as  possible  ordnance  to  be  dug,  and  did  not  list  any  of  them  as  clutter.  The  missing 
ordnance  are  three  of  the  four  20  mm  targets  and  one  81  mm  target.  The  percentage  of  the  to-be- 
dug  items  that  were  in  fact  nonordnance  was  large  (83%),  as  expected,  since  the  magnetometer 
array  and  matched  filter  analysis  provide  detection,  but  only  a  limited  discrimination  capability. 


Table  4,  Matched  Filter  AutoProcessor  Dig  List  Results  for  JPG  TD  Area  3. 


Targets  Detected 

Targets 

Buried 

UXO 

Detections 
and  Pd 

Probability 

of 

Detection 

UXO  Not 
Detected 

Probability  of 
False  Alarm 

20  mm 

assumed 

present 

55 

20  UXO 

35  OE 

16  of  20 

80% 

3  (20  mm) 

1  (81  mm) 

83% 

no  20  mm  assumed 
present 

55 

16  UXO 

39  OE 

15ofl6 

94% 

1  (81  mm) 

83% 

It  is  clear  that  the  relatively  low  detection  rate  (80%)  is  mostly  due  to  problems  detecting  the  20 
mm  targets. 
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Of  the  three  non-detected  20  mm  targets: 


•  the  first  was  loeated  in  a  data  gap  at  even  the  shallowest  ehosen  filter  depth  of 
0.10  m  and  was  not  found  by  the  matehed  filter 

•  the  seeond  was  found  by  the  matched  fdter  but  screened  out  because  it  had  a  low 
data/model  coherence  (0.55,  the  threshold  was  0.80) 

•  the  third  was  not  found  by  the  matched  filter,  and  was  not  seen  when  the  data  was 
inspected  visually. 

The  missed  8 1  mm  was  found  by  the  matched  filter  peak  finder.  However,  it  was  at  the  edge  of  a 
data  gap;  this  reduced  the  data/model  coherence  to  0.74,  falling  below  the  threshold  because  a 
substantial  part  of  its  signature  was  in  the  data  gap  and  could  not  be  fit. 

Table  5  shows  the  accuracy  of  the  target  locations  compared  to  the  ground  truth  for  JPG  TD 
Area  3. 


Table  5,  Horizontal  Position  and  Depth  Differences  between  Detected  UXO  and  Ground 

Truth  for  JPG  TD  Area  3, 


Horizontal  Difference  (m) 

Depth  Difference  (m) 

average 

0.10 

0.14 

median 

0.09 

0.11 

4,3  TECHNOLOGY  COMPARISON 

Our  dig  lists  for  all  three  JPG  TD  areas,  as  well  as  the  ground  truth  for  Area  3  (provided  by 
George  Robitaille  of  the  U.S.  Army  Environmental  Center),  are  appendices  in  the  Final  Report  , 
but  the  MFAP  target  detections  and  dig  list  are  compared  with  the  ground  tmth  and  with  the 
results  of  other  UXO  detection  systems  and  analyses  below. 

Table  6  shows  the  detection  probabilities  and  false  alarm  counts  for  MFAP  and  U-Hunter 
analyses.  These  results  are  shown  graphically  in  Figure  7,  along  with  the  Receiver  Operating 
Characteristics  (ROCs)  for  the  primary  demonstrators  using  electromagnetic  induction  (EMI) 
systems.  The  three  colored  points  on  each  plot  indicate  the  relative  performance  of  analysis  of 
magnetometer  data  using  the  MFAP  and  U-Hunter,  as  well  as  standard  Mag  and  Flag 
methodology.  U-Hunter  and  Mag  and  Flag  points  generally  lie  far  to  the  right  of  the  curves, 
indicating  significantly  poorer  performance.  The  MFAP  points,  on  the  other  hand,  reveal 
performance  comparable  to  the  best  demonstrators  in  Area  2,  and  intermediate  between  the 
various  demonstrators  in  Areas  1  and  3. 
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Table  6,  Detection  Probabilities  and  False  Alarm  Counts  for  U-Hunter  and  MFAP 

Analyses  for  JPG  TD  2000, 


U-Hunter 

MFAP 

Pd 

false  alarms 

Pd 

false  alarms 

Area  1 

0.72 

165 

0.55 

52 

Area  2 

0.7 

200 

0.6 

62 

Area  3 

0.65 

124 

0.65 

78 

0  20  40  60  80  100  120  140  160  180  200  220 


Falsa  Alarm  Count 


False  Alarm  Count 


EMI  Sensors  Magnetometers 
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Matched  Filter 
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▲ 
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NAEVA 

♦ 

MAG  &  Rag 

Figure  7.  ROC  Curves  for  JPG  TD  Tested  Systems  (NRL,  Geophex  and  NAEVA)  and  Detection 
Statistics  for  Three  Magnetometer  Analysis  Methods  (MFAP,  U-Hunter,  MAG  &  Flag). 
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5.0 


COST  ASSESSMENT 


5.1  TIME  AND  EXPERIENCE  EVALUATION 

We  investigated  the  ease  of  use  of  the  MFAP,  in  terms  of  how  much  operator  training  is 
necessary  to  obtain  optimal  results.  An  operator  with  no  previous  experience  with  magnetic 
systems  and  no  technical  knowledge  of  ordnance  detection  systems  was  given  about  20  minutes 
of  basic  training  on  running  the  MTADS  DAS  and  picking  targets  from  magnetometer  data.  He 
then  used  the  DAS  to  analyze  Area  3  of  JPG  TD.  The  results  are  shown  in  Table  7.  There  was  a 
learning  curve  for  picking  targets  with  the  DAS;  he  analyzed  the  data  three  times  (Trials  1,  2  and 
3),  picking  more  targets  each  time,  as  shown  in  the  table. 

This  operator  then  was  given  a  brief  training  (15  minutes)  on  how  to  run  the  MFAP,  for  which  he 
used  all  default  values  for  thresholds  and  just  ran  the  peak  picker.  (The  raw  data  had  previously 
been  filtered  at  12  depths.)  He  ran  the  peak  picker  on  these  12  depths.  Finally,  he  ran  the  peak 
picker  on  the  5  depths  (described  in  section  4.1)  that  had  been  analyzed  previously  by  an 
experienced  operator.  The  inexperienced  operator  achieved  a  75%  (MFAP)  or  80%  {MTADS 
DAS)  detection  rate. 

These  results  are  compared  in  Table  7  to  those  of  an  experienced  operator  using  both  the  MTADS 
DAS  and  the  MFAP.  The  experienced  operator  using  MTADS  found  all  targets  except  one  20 
mm,  and  using  MFAP  found  all  targets  except  two  20  mm.  Using  the  MFAP,  the  experienced 
operator  did  more  than  autoprocess;  he  ran  the  automatic  peak  finder  but  then  examined  the 
images  to  decide  if  any  dipole-like  targets  were  missed.  He  still  missed  an  additional  20  mm 
compared  to  MTADS  because  it  was  lying  very  near  a  data  gap.  (This  weakness  of  the  MFAP  is 
discussed  further  in  Section  6.0.) 

Note  that  detection  rates  in  Table  7  refer  to  targets  identified  in  the  first  data  analysis,  before 
screening.  The  final  dig  list  excludes  some  of  these  detected  targets  because  of  low  data/model 
coherence  (due  to  data  gaps). 

We  also  investigated  the  processing  and  analysis  times  for  the  Matched  Filter  AutoProcessor  and 
compared  it  to  analysis  times  using  the  MTADS  Data  Analysis  System  (DAS).  Our  experience 
with  the  existing  automatic  processor  in  the  MTADS  DAS  is  that  it  can  process  5  to  10  acres  per 
hour  of  magnetometer  array  data,  depending  on  the  threshold  and  target  density.'^  By 
comparison,  a  human  operator  can  process  about  1  acre  per  hour  in  the  user-interactive  mode  at 
the  same  target  density.  The  analysis  time  using  the  MTADS  DAS  is  shown  below  in  Table  8  for 
the  inexperienced  operator’s  third  trial  (refer  to  Table  7).  Although  the  Matched  Filter 
Autoprocessor  (MFAP)  data  analysis  of  JPG  TD  was  performed  in  AETC’s  offices  and  not  in  the 
field,  we  tracked  the  processing  times  for  the  various  steps  involved  in  the  MFAP;  running  the 
filter  at  a  given  depth  and  selecting  the  peaks.  These  are  shown  in  Table  8.  Note  that  the  Pre- 
Processing  Times  are  for  computations  run  completely  automatically,  without  any  required 
operator  intervention  or  presence.  The  analysis  times  are  for  comparable  operator  effort. 
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Table  7.  MFAP  and  MTADS  Performance  (before  attribute  sereening)  by  Inexperienced 
and  Experienced  Operators  on  Area  3  of  JPG  TD,  Assuming  20  mm  Ordnance  Was 

Present, 


Analysis  System 

#  of  targets 
picked 

ordnance 

detected 

Inexperienced 

Operator 

Experienced 

Operator 

ordnance 

missed 

Pd 

ordnance 

missed 

Pd 

MTADS  DAS, 

Trial  1 

68 

12  of  20 

4  20mm 

1  81  mm 

2  60  mm 

1  155  mm 

65% 

1  20  mm 

95% 

MTADS  DAS, 

Trial  2 

128 

16  of  20 

3  20  mm 

1  81  mm 

80% 

1  20  mm 

95% 

MTADS  DAS, 

Trial  3 

144 

16  of  20 

3  20  mm 

1  81  mm 

80% 

1  20  mm 

95% 

MFAP 

(12  depths) 

1091 

15  of  20 

2  20  mm 

1  81  mm 

2  60  mm 

75% 

2  20  mm 

90% 

MFAP 
(5  depths) 

561 

14  of  20 

3  20  mm 

1  81  mm 

2  60  mm 

70% 

2  20  mm 

90% 

Table  8,  Processing  Times  for  an  Inexperienced  Operator  Using  the  MTADS  DAS  and 
MFAP  Analysis  Systems  on  Data  from  Area  3  of  JPG  TD. 


Analysis  System 

Pre-Processing  Time*  (min) 

Analysis  Time** 

(min) 

MTADS  DAS 

n/a 

105 

MFAP  (12  depths) 

491 

50 

MFAP  (5  depths) 

166 

20 

*  The  pre-processing  time  is  the  time  to  run  the  matched  filter  on  the  raw  data  for  all  selected  depths  to 
produce  the  matched  filter  output  images.  This  time  is  both  filter-depth  dependent  and  computer-platform 
dependent.  On  the  computer  at  AETC,  it  took  9  minutes  for  the  shallowest  depth  and  88  minutes  for  the 
deepest  depth. 

**  The  analysis  time  is  the  time  to  operate  the  automatic  peak  finder  on  the  matched  filter  output  image. 
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6.0  IMPLEMENTATION  ISSUES 


A  number  of  issues  appeared  during  the  development  and  testing  of  the  MFAP.  Some  were 
solved;  others  were  beyond  the  scope  of  the  project.  Both  types  are  listed  below,  along  with 
solutions  found  or  suggestions  for  the  future. 

(1)  The  CDF  threshold/contouring  procedure  used  for  automated  target  picking,  which  worked 
on  the  previous  sites,  produced  an  overwhelming  number  of  spurious  targets  when  applied 
to  the  JPG  TD  data.  We  decided  that  using  the  Cumulative  Distribution  Function  of  the 
filter  output  over  an  entire  image  was  a  major  cause  of  the  problem.  Calculating  CDFs  and 
picking  targets  over  smaller  sub-areas  of  an  image  has  eliminated  this  problem  and  the 
target-picking  algorithm  has  been  appropriately  modified. 

(2)  The  matched  filter  procedure  is  sensitive  to  survey  defects  that  produce  missing  data. 
Several  UXO  targets  were  not  detected  or  included  in  the  dig  list  because  of  the  expansion 
of  areas  of  missing  data  with  increasing  filter  depth.  One  solution  might  be  to  relax  the 
data/model  coherence  threshold  for  objects  very  near  the  edge  of  data  gaps. 

A  solution  to  the  problem  of  data  lost  at  the  edges  of  the  survey  region  is  to  survey  outside 
the  search  area.  Since  the  filter  box  width  for  a  filter  depth  of  3.5  m  is  14  m,  one  would 
need  to  survey  at  least  7  m  beyond  the  edges  of  the  UXO  search  area  to  avoid  missing 
targets  as  deep  as  3.5  m  (a  7.7  m  border  region  is  required  for  4  m  deep  targets). 

For  interior  gaps,  man  portable  systems  will  decrease  the  gaps  but  not  eliminate  them.  One 
can,  however,  interpolate  from  the  edges  of  an  interior  gap  towards  its  center  so  that  the 
data  is  filled  in.  Obviously,  any  interpolation  scheme  must  give  reasonable  results  across 
these  gaps  or  we  risk  trading  the  problem  of  missing  data  for  that  of  spurious  or  non¬ 
physical  data.  To  develop  a  reasonable  interpolation  procedure,  gaps  of  exactly  the  same 
size  and  shape  as  real  regions  of  missing  data  should  be  introduced  into  areas  where  we  do 
have  data.  We  can  then  compare  interpolated  results  with  the  original  removed  data  to  see 
which  interpolation  procedures  best  reproduce  the  original.  This  scheme  has  not  been 
developed  in  this  project. 

(3)  When  we  compared  Area  3  results  with  the  ground  truth,  we  found  we  were  too 
conservative  in  setting  our  dig/no-dig  boundary  of  fitted  dipole  orientation;  this  was 
changed  for  the  analysis  of  the  other  two  areas. 

(4)  Operating  the  MFAP  on  magnetometer  array  data  provides  less  discrimination  capability 
than  operating  on  active  electromagnetic  data.  Incorporation  of  the  matched  filter  into 
active  EM  data  analysis  systems  should  provide  further  screening  criteria  and  thus  better 
performance. 
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APPENDIX  B 


OASIS  MONTAJ  INTEGRATION 

B  1.0  OASIS  MONTAJ  INTEGRATION  OVERVIEW 

The  matched  filter  algorithm,  which  was  originally  codified  in  the  Interactive  Data  Language 
(IDL;  Research  Systems,  Inc.),  implements  optimal  linear  filtering  in  a  threshold-based  UXO 
detection  scheme.  Following  successful  demonstration  of  the  approach,  the  algorithms  were 
recoded  to  seamlessly  integrate  with  OASIS  Montaj.  OASIS  Montaj  is  developed  and  marketed 
commercially  by  Geosoft,  Inc.,  and  includes  a  very  large  capacity  database,  a  graphical  user 
interface,  and  a  plethora  of  geophysical  processing  and  mapping  routines. 

As  described  above,  the  matched  filter  processor  convolves  a  dipole-based  model  signature  with 
a  gridded  (or  krigged)  approximation  of  the  measured  data.  As  such,  the  inputs  include  the 
gridded  data  file,  geographic  information  regarding  the  site  (required  in  order  to  calculate  the 
inclination  and  declination  of  the  Earth’s  magnetic  field),  the  size  of  the  filter  box  (described 
below),  and  the  distance  between  the  sensors  and  the  ground’s  surface.  The  outputs  include  a 
filtered  output  grid  and  an  OASIS  Montaj  database  containing  the  output  grid  data,  spatial 
locations,  and  model  parameters. 

There  are  three  basic  steps  involved  with  the  OASIS  Montaj  matched  filter  executable.  First,  the 
input  data  are  submitted  to  the  filtering  routine  by  calling  a  Geosoft  executable 
(‘matchfilter.gx’).  The  ‘matchfilter.gx’  interfaces  with  the  OASIS  Montaj  database  and  passes 
the  data  to  a  dynamic  link  library  that  creates  the  filter  output  grid  and  model  parameters.  The 
second  step  involves  refining  the  filter  output  (if  desired)  by  utilizing  the  model  parameters.  In 
the  third  step,  peaks  are  identified  in  the  final  data  to  create  the  anomaly  list. 

Figure  B-1  shows  the  user  dialogue  window  that  is  displayed  upon  calling  the  Geosoft 
executable  (‘matchfilter.gx’).  Geosoft  executables  are  processes  that  perform  a  variety  of  data 
processing  tasks.  Required  inputs  for  the  matched  filter  executable  include  an  OASIS  Montaj 
grid  file,  the  sensor  offset  (distance  between  the  ground’s  surface  and  the  magnetic  sensors),  the 
base  length  of  the  filter  box,  the  latitude  and  longitude  of  the  site,  and  geomagnetic  reference 
field  information.  The  ‘Output  Grid  File’  requests  a  filename  for  the  matched  filter  output 
results.  A  database  containing  all  measured  data  and  fit  results  (or  model  parameters)  is  also 
created  using  the  same  name  but  with  a  *.gdb  extension. 

Similar  to  the  previously  prototyped  IDF  code,  a  fdter  box  is  created  around  each  point  (xq,  yo) 
in  the  grid.  The  model  signal  in  the  matched  filter  is  a  dipole  located  at  the  center  of  the  filter 
box  (xo,  yo)  for  various  depths,  plus  a  linear  magnetic  background  field.  A  multiple  linear 
regression  on  the  data  for  all  points  in  the  box  yields  the  best-fit  signal  model  dipole  and  the 
background  field  constants. 
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Figure  B-1,  Screen  snapshot  of  the  Matched  Filter  Dialogue  box. 

The  OASIS  Montaj  version  of  the  Matched  Filter  searches  over  six  target  depths  (0.25m,  0.5m, 
0.75m,  1.0m,  1.25m,  and  1.5m).  The  depth  that  produces  to  the  best  fit  (i.e.,  lowest  error  in 
terms  of  model  minus  measured  data)  is  selected,  and  the  model  parameters  for  that  depth  are 
stored.  An  additional  modification  was  made  regarding  the  size  of  the  filter  box.  In  the  IDL 
prototype  code,  the  size  of  the  filter  box  was  a  function  of  the  user-selected  interrogation  depth. 
It  was  concluded,  however,  that  the  matched  filter  output  was  not  sensitive  to  the  size  of  the  filter 
box.  In  the  OASIS  Montaj  version,  the  size  of  the  filter  box  is  assigned  by  the  user  and  is  not  a 
function  of  the  interrogation  depth. 

As  part  of  the  JPG  TD  program,  the  NRL  surveyed  the  three  prepared  areas  using  the  vehicular 
Multisensor  Towed  Array  Detection  System  (MTADS)  magnetometer  arrays.  These  data  were 
preprocessed  by  the  NRL  to  create  geo-referenced  mapped  data  files.  The  MTADS 
magnetometer  data  from  Area  1  was  analyzed  using  the  matched  filter  routine  developed  for  the 
OASIS  Montaj  environment. 
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B  2.0  OASIS  MONTAJ  FILTER  PERFORMANCE 


B  2.1  SYNTHETIC  DATA 

Synthetic  data,  created  using  forward  models,  were  used  to  evaluate  the  OASIS  Montaj- 
embedded  matched  filter  routine.  Table  B-2  provides  details  regarding  the  location  and 
characteristics  of  magnetic  dipoles  while  Figure  B-2  presents  a  color-coded  contoured 
representation  of  the  synthetic  data.  The  synthetic  data  contain  nine  distinct  dipoles,  each  with  a 
unique  combination  of  depth,  location,  and  orientation. 

Table  B-2.  Location  and  Characteristics  of  Magnetic  Dipoles  for  Synthetic  Data 


ID 

X(m) 

Y(m) 

Depth  (m) 

Inc (deg) 

Dec  (deg) 

Size  (m) 

1 

10 

10 

0.5 

0 

0 

0.05 

2 

20 

10 

0.5 

0 

90 

0.05 

3 

30 

10 

0.5 

0 

180 

0.05 

4 

10 

20 

0.5 

30 

0 

0.05 

5 

20 

20 

1.5 

30 

90 

0.10 

6 

30 

20 

0.5 

30 

180 

0.05 

7 

10 

30 

0.5 

60 

0 

0.05 

8 

20 

30 

0.5 

60 

90 

0.05 

9 

30 

30 

0.5 

60 

180 

0.05 
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Figure  B-2.  Color-coded  map  of  synthetic  data  used  to  test  the  matched  filter  routine. 
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The  matched  filter  output  data,  shown  in  Figure  B-3,  clearly  indicate  the  presence  of  nine 
dipoles.  Processing  artifacts  are  also  apparent.  The  artifacts  occur  when  the  filter  window 
contains  only  a  portion  of  the  dipolar  signal  (typically  a  single  positive  or  negative  lobe)  that  is 
associated  with  the  causative  source.  When  this  happens,  the  filter  incorrectly  creates  the 
missing  lobe,  which  produce  erroneous  output  values.  These  artifacts  were  present  in  the  IDL 
prototyped  code  as  well. 
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Figure  B-3.  Color-coded  map  showing  results  of  the  OASIS  Montaj  Matched  Filter, 
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The  OASIS  Montaj  matched  filter  stores  the  filter  results  and  all  derived  model  parameters  in  a 
database.  The  model  parameters  can  be  used  to  refine  the  filter  output.  Figure  B-4  shows,  in 
profile  form,  the  matched  filter  output  (‘MFoutput’),  the  model  error  (‘Chisq’),  and  the  ratio 
(‘MFoutput/Chisq’).  When  the  fdter  window  coincides  with  the  center  of  a  dipolar  signal,  the 
error  term  is  dramatically  reduced.  In  other  words,  the  derived  model  closely  matches  the  input 
(measured  or  synthetic)  data  when  it  is  directly  on  top  of  the  center  of  the  dipole.  The  horizontal 
axis  in  Figure  B-4  represents  the  sample  number  (or  fiducial).  For  reference,  fiducial  number 
12,960  coincides  with  x=20,  y=20  in  Figures  B-2  and  B-3.  There  are  three  major  groupings  in 
Figure  B-4  -  each  with  multiple  local  peaks.  The  left-most  grouping  represents  data  along 
y=  19.75m.  The  center  grouping  represents  data  along  y=20.0  meters,  and  so  on.  Within  the 
center  grouping  (y=20.0  meters),  there  are  three  peaks  in  the  MFoutput  channel  (blue)  -  each 
peak  represent  one  of  the  three  dipoles  located  at  x=10-,  20-,  and  30-meters. 


Figure  B-4.  Profile  view  of  the  Matched  Filter  output  and  Error  term.  Fiducial  number  12,960 
corresponds  to  x=20,  y=20  in  Figures  B-2  and  B-3.  Note  that  the  error  term  (‘Chisq’)  is  minimized 
when  the  filter  output  ‘MFoutput’  is  maximized.  In  other  words,  when  the  center  of  the  filter  box 
coincides  with  a  dipole  source,  the  model  data  closely  match  the  synthetic  data. 

Figure  B-5  shows  the  results  of  dividing  the  matched  filter  output  by  the  error  term.  As  observed 
in  the  figure,  the  anomalies  possess  a  very  high  signal-to-noise  ratio  and  the  processing  artifacts 
are  reduced. 

The  derived  model  parameters  for  these  synthetic  data  are  shown  in  Table  B-3.  Because  the 
OASIS  Montaj  version  of  the  matched  filter  routine  includes  searches  over  multiple  depths,  the 
anomaly  depth  estimates  are  reasonable.  Comparing  Tables  B-2  and  B-3  reveals  that  the  size, 
depth,  inclination,  and  declination  estimates  are,  in  fact,  quite  good. 
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Figure  B-5.  Color-coded  map  of  the  matched  filter  output  divided  by  the  error  term. 
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Table  B-3,  Matched  Filter  Results  and  Derived  Model  Parameters 


ID 

X(m) 

Y(m) 

Depth  (m) 

Inc (deg) 

Dec  (deg) 

Size  (m) 

1 

10 

10 

0.5 

0.5 

0.1 

0.05 

2 

20 

10 

0.5 

-0.3 

90.0 

0.05 

3 

30 

10 

0.5 

-0.5 

180.0 

0.05 

4 

10 

20 

0.5 

30.7 

0.5 

0.05 

5 

20 

20 

1.5 

29.6 

90.6 

0.10 

6 

30 

20 

0.5 

29.4 

179.4 

0.05 

7 

10 

30 

0.5 

61.0 

1.3 

0.05 

8 

20 

30 

0.5 

59.3 

91.5 

0.05 

9 

30 

30 

0.5 

59.2 

178.7 

0.05 
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The  data  gaps,  which  are  indicated  in  Figure  B-6  by  holes  in  the  sensor  track  density,  create 
problems  for  the  matched  filter  algorithm.  The  default  gridding  process  used  to  create  the  map 
shown  in  Figure  B-7,  for  example,  did  not  allow  the  gaps  to  be  interpolated.  These  data  gaps  in 
the  gridded  data  will  be  enlarged  by  one -half  the  size  of  the  filter  window  during  the  matched 
filtering  process.  As  a  result,  anomalies  near  the  margin  of  data  gaps  will  be  missed  if  the  data 
gaps  are  not  interpolated  during  the  gridding  process. 
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Figure  B-7.  Color-coded  map  of  the  MTADS  magnetic  data,  Area  1  JPG  TD, 
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By  allowing  data  gaps  to  be  interpolated  during  the  gridding  proeess,  the  problems  assoeiated 
with  data  gaps  are  eliminated.  OASIS  Montaj  allows  the  user  to  define  the  distance  over  which 
grid  values  are  interpolated.  Figure  B-8  presents  a  map  of  Area  1  with  the  data  gaps  filled  in 
(i.e.,  interpolated)  during  gridding. 
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Figure  B-8.  Color-coded  map  of  the  MTADS  magnetic  data,  Area  1  JPG  TD,  Note  that  the  data 

gaps  apparent  in  Figures  B-6  and  B-7  have  been  interpolated. 
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Figures  B-9  through  B-16  present  results  of  the  OASIS  Montaj  matched  filter  data  from  Area  1. 
The  various  maps  are  derived  from  the  database  that  contains  the  filter  output  and  model 
parameters  that  were  derived  during  filtering. 

The  series  of  maps  and  the  associated  logic  presented  below  differs  from  the  IDL  prototyped 
matched  filter  procedures.  The  IDL  prototyped  matched  filter  algorithm  did  not  search  over 
multiple  depths  and  did  not  store  the  model  parameters.  Instead,  the  filter  algorithm  was 
repeatedly  run  at  various  depths  and  the  derived  fdter  images  were  each  examined  for  peaks  (i.e., 
anomalies).  The  anomaly  lists  were  subsequently  combined  into  a  master  list.  Once  the  master 
anomaly  list  was  created,  the  measured  data  (not  the  gridded  data)  around  each  anomaly  location 
was  submitted  to  a  dipole-estimating  algorithm.  The  dipole-estimating  algorithm  analyzed  each 
target  and  stored  the  best  fitting  model  parameters.  The  anomalies  were  then  screened  and  sorted 
based  upon  the  derived  model  parameters. 

The  process  presented  herein  basically  tries  to  enhance  or  retain  anomalies  that  are  observed  in 
the  raw  magnetic  data  while  reducing  or  eliminating  those  features  that  are  not  associated  with 
expected  signatures.  All  of  the  model  parameters  were  derived  from  the  gridded  approximation 
of  the  measured  data. 
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Figure  B-9  presents  the  matched  filter  output  of  Area  1,  JPG,  without  any  post  processing.  As 
observed  in  the  figure,  a  number  of  anomalies  possess  very  large  amplitudes  while  others  do  not 
appear  to  be  caused  by  an  isolated  item  (i.e.,  they  are  not  localized  and  appear  to  be  related  to  the 
background  geology  -  compare  Figures  B-8  and  B-9). 
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Figure  B-9.  Color-coded  map  showing  the  results  of  the  matched  filter,  Area  1  JPG  TD. 
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Figure  B-10  presents  the  ratio  of  the  matehed  filter  output  and  the  derived  model  error  term. 
This  ‘ratio’  map  is  notably  different  in  nature  than  that  shown  in  Figure  B-9.  Unlike  the  noise- 
free  synthetic  data  case  presented  earlier,  the  presence  of  complicated  structures  in  the  measured 
response  associated  with  nearby  targets,  clutter,  or  geologic  sources  produces  numerous  potential 
targets. 
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Figure  B-10.  Color-coded  map  showing  the  results  of  dividing  the  matched  filter  output  by  the 

derived  model  error;  Area  1  JPG  TD. 
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Based  on  a  visual  inspection  of  the  filter  output  and  raw  data,  it  was  noted  that  realistic 
anomalies  observed  in  the  raw  data  also  had  filter  output  values  larger  than  five.  Figure  B-11 
presents  the  ratio  of  the  matched  filter  output  and  the  derived  model  error  term  for  those  filter 
values  that  are  greater  than  five.  [Note:  The  output  filter  values  depend  on  the  filter  box  size  -  a 
filter  box  with  a  base  length  of  3.75m  was  used  for  this  example.  It  is  incorrect  to  assume  that 
different  data  sets  should  apply  the  same  threshold.] 


Figure  B-11.  Color-coded  map  showing  the  results  of  dividing  the  matched  filter  output  values  that 

are  larger  than  five  by  the  derived  model  error;  Area  1  JPG  TD. 
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Discarding  those  locations  that  had  a  reported  depth  (depth  is  one  of  the  fitted  parameters)  of 
greater  than  one  meter  and  an  apparent  size  (apparent  size  is  also  one  of  the  fitted  parameters)  of 
less  than  0.13  m  further  restricted  the  data  presented  in  Figure  B-12.  These  data  were  then  used 
as  the  final  filter  output.  Potential  anomalies  were  selected  using  the  OASIS  Montaj  grid  peak 
utility. 
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Figure  B-12.  Color-coded  map  showing  the  final  post-processed  results  of  the  matched  filter; 

Area  1  JPG  TD.  See  text  for  discussion. 
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Figure  B-13  presents  the  final  post-proeessed  matehed  filter  output  grid  overlain  by  the  ground 
truth  (+  symbols  and  text)  and  the  anomalies  selected  by  the  grid  peak  executable  (circles; 
minimum  peak  value  of  0.3).  Utilizing  the  fitted  model  parameters  to  improve  the  filter  output 
(as  described  above)  revealed  1 1  targets  that  are  not  apparent  in  the  unprocessed  filter  output 
(compare  Figures  B-13  and  B-14).  The  additional  detected  targets  included  57mm,  60mm,  and 
81mm  items. 
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Figure  B-13.  Color-coded  map  showing  the  final  post-processed  results  overlain  by  the  ground 

truth  (+)  and  anomaly  picks  (circles);  Area  1  JPG  TD, 
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Figure  B-14,  Color-coded  map  showing  the  original  matched  filter  output  overlain  by  the  ground 
truth  (+  and  text)  and  anomaly  picks  (circles);  Area  1  JPG  TD.  Comparing  these  data 
with  that  shown  in  Figure  B-13  indicates  that  utilizing  the  model  parameters  to  post 
process  the  filter  results  identified  an  additional  11  emplaced  targets. 
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The  anomalies  identified  after  sereening  using  the  IDL  prototyped  filter  are  superimposed  (using 
square  symbols)  on  the  final  processed  OASIS  Montaj  matched  filter  results  in  Figure  B-15. 
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Figure  B-15.  Color-coded  map  showing  the  final  post-processed  results  overlain  by  the  ground 
truth  (+),  OASIS  Montaj  matched  filter  anomalies  (circles),  and  anomaly  picks  (after 
screening)  using  the  IDL  matched  filter  algorithm  (squares);  Area  1  JPG  TD. 
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In  summary,  a  total  of  150  anomalies  were  identified  for  Area  1  using  the  OASIS  Montaj 
embedded  matehed  filter.  Table  B-4  presents  a  comparison  of  the  detected  versus  emplaced 
UXO  items  in  Area  1 . 


Table  B-4,  Detected  versus  Emplaced  UXO  items  in  Area  1,  JPG  TD 


Ordnance 

Number 

Empiaced 

Type 

Identified 

20mm 

0 

10 

57mm 

2 

5 

60mm 

5 

5 

76mm 

3 

3 

81mm 

4 

5 

105mm 

2 

2 

152mm 

2 

2 

155mm 

2 

2 

4.2  inch 

2 

2 

5  inch 

3 

3 

2.75  inch 

4 

4 

Total 

29 

43 

For  these  data,  the  detection  rate  was  67%  primarily  because,  as  noted  in  Table  B-4,  none  of  the 
20mm  items  produced  peak  values  above  the  selected  threshold  of  0.3.  This  is  not  unexpected 
based  on  our  previous  experience  with  the  IDL  prototyped  matched  filter  and  the  almost 
negligible  measured  response  associated  with  the  20mm’s  (Figure  B-16).  If  the  20mm  items  are 
excluded,  the  detection  rate  increases  to  88%.  The  three  missed  57mm  items  were  located  near 
magnetically  active  regions  and  probably  would  not  be  detected  visually  (Figure  B-16).  The 
missed  81mm  item  was  also  located  within  an  active  magnetic  area  and  is  not  readily  apparent 
when  the  raw  magnetic  data  are  inspected  visually  (Figure  B-16). 

The  percentage  of  items  declared  as  potential  targets  by  the  matched  filter  algorithm  that  are  not 
emplaced  ordnance  items  is  approximately  80%.  This  is  similar  to  number  of  false  alarms 
reported  during  the  previous  IDL  development. 
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Figure  B-16  presents  the  raw  magnetie  data  overlain  by  the  identified  anomalies  from  the  OASIS 
Montaj  embedded  routine  (circles)  and  the  ground  truth  (+  symbol  and  text).  Although  some  of 
the  matched  fdter  target  selections  would  probably  be  discarded  after  visually  inspection,  we  did 
not  do  so  to  preserve  this  evaluation  of  the  filtering  approach. 
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Figure  B-16.  Color-coded  map  showing  the  raw  magnetic  data  overlain  by  the  ground  truth  (+) 
and  anomalies  identified  by  the  OASIS  Montaj  embedded  matched  filter  routine 

(circles);  Area  1  JPG  TD, 
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